Multiple Redundant Services with Reputation

ABSTRACT

Multiple copies of web services reside on associated computing devices, each having an associated reputation. A client may desire to access the web service having the highest or best reputation to be ensured of a greater degree of accuracy and confidence. The client does a search, and attaches to whichever web service has the highest reputation. By running multiple copies of the web services, they may vote amongst themselves on the results in the event that one or more of the services starts giving incorrect or otherwise inconsistent results. Combining the voting with reputation data associated with each copy of the web service allows a service&#39;s reputation to be dynamically adjusted based upon how faithfully it computes the results of work items sent to it.

RELATED APPLICATIONS

This application is a divisional of, and claims priority to, U.S. patentapplication Ser. No. 10/909,139, filed Jul. 30, 2004 and titled,“Multiple Redundant Services with Reputation”, the disclosure of whichis incorporated herein, in its entirety.

FIELD OF THE INVENTION

The present invention relates generally to the field of informationmanagement, and, more particularly, to the identification and executionof multiple web services with reputation data.

BACKGROUND OF THE INVENTION

There are many types of computing services, resources and data thatcomputer users and applications need to manage and otherwise access.Such resources and data include services and data maintained locally,and data maintained on corporate networks and other remotely accessiblesites including intranets and the internet.

The concept of web services is generally directed to providing acomputing service to clients via protocols and standards that arecross-platform in nature. In general, web services provides the basictools for wiring the nodes of a distributed application together,regardless of the type of platform on which the requesting client isrunning. As there are many different computing platforms, variousplatform-independent mechanisms and protocols that facilitate theexchange of network information are becoming commonplace in webservices, including HTTP (HyperText Transfer Protocol), XML (eXtensibleMarkup Language), XML Schema, and SOAP (Simple Object Access Protocol)XML. Traditional web services, in which businesses, organizations, andother providers offer services to users and applications, are presentlybased on these standards. Moreover, a web service can represent othertypes of resources, such as hardware devices, databases, and the like.

Currently, the protocols by which web services exchange data with aclient allow a client to send a message to a server port (e.g., URI)corresponding to the service and receive a message back at the client,in a manner agreed on according to a contract provided by the webservice.

Reputation management systems are well-known in peer-to-peer networks,which provide a means of distributing many kinds of data, such as music,movies, and software. Given the open nature of these networks (that is,the fact that they allow anyone on the network to share any type andnumber of files), peer-to-peer networks are prone to abuses by anonymousmalicious peers who share inauthentic files. The presence of thesemalicious peers can lead to the network becoming inundated with viruses,spam, and other inauthentic files. Recognizing this problem has ledresearchers to propose reputation management systems for these networks.These reputation management systems are algorithms that allow individualpeers on a peer-to-peer network to rate each other according to thequality of their past transactions with each other. Once a peer has beenrated, its rating can be used by other peers to help them find the bestsources of good, authentic files and keep the effects of malicious peerson the network to a minimum.

There are several reputation management systems that are currently beingused in situations such as online auctions and file sharing. eBay is awebsite where buyers and sellers can meet and conduct auctions entirelyonline. In eBay's reputation management system, buyers and sellers canrate each other based on their past transactions with each other. Thisis done via users' leaving comments on one another's eBay user pages,which can be positive, neutral, or negative. An eBay member's reputationis calculated by assigning each type of comment a point value (+1 pointfor positive comments, 0 points for neutral comments and −1 point fornegative comments) and summing the point values of all of a member'stransactions to obtain that member's “reputation,” which is the numberof positive comments that the user has received. An eBay user'sreputation score can then be used by other users as a factor in theirdecisions on whether to conduct a transaction with that user or not.

Kazaa is a peer-to-peer file sharing application that allows its usersto share files with other Kazaa users. It also allows users to searchfor and download files from other users who are sharing them. Kazaa hasimplemented a reputation management system consisting of two components,“Integrity Rating” and “Participation Levels.” Integrity Rating allowspeers who share files to rate their own files in terms of theirtechnical merits, such as whether the files have accurate metadata andare of high quality. In order to guide other peers toward thehighest-quality files available for download, users are encouraged toIntegrity Rate their files and delete files that should not be shared(e.g., virus-infected files or files that are corrupted).

In Kazaa's reputation management system, each peer has a “ParticipationLevel” that is based on the quality and amount of files that it shares.A peer's Participation Level is a number that reflects the ways in whichthat peer has used Kazaa to upload and download files. A peer'sParticipation Level is meant to reward peers who share many IntegrityRated files by providing those peers with increased bandwidth that theycan use to download files from other peers on the network.

There are several other proposed reputation management systems, such asDebit-Credit Reputation Computation (DCRC), Credit-Only ReputationComputation (CORC), and Distributed Polling (Xrep). DCRC works bycrediting peers' reputation scores for serving content and debits themfor downloading. CORC credits peers' reputation scores for servingcontent, but does not debit them. Under CORC, the expiration of a peer'sreputation score serves as a debit. In DCRC and CORC, peers can alsoearn credit for staying online and processing query requests sent tothem by other peers. Since DCRC and CORC allow peers to store their ownreputation scores, these reputation management schemes use a centralReputation Computation Agent which is contacted periodically by thepeers to earn credit for their contribution to the network.

Distributed Polling (Xrep Protocol) determines whether peers aretrustworthy based on votes that peer i takes regarding other peers'experiences with each peer j who responds to i's queries for content. InXrep, each peer maintains a resource repository, which is a table thatassigns a binary value describing whether a given resource is good (+)or bad (−). Each peer also has a servant repository, which is a tablethat keeps track of each servant, or other peer, that a peer hasinteracted with and records the numbers of successful and unsuccessfuldownloads from that peer.

When peer i receives responses to a search query from other peers, itwill find the best download source for the file that it is looking forby polling its peers for their opinions of the resource and the peersthat offer it. After receiving a set of votes from the other peers, iwill discard any votes that have been tampered with and cluster theremaining votes by their IP addresses. i will then weight the voteswithin each cluster, such as by taking an average of the cluster's votesweighted on the size of the cluster. Next, i will contact the mostreliable peer that hosts the file (based on the votes that i hasreceived) to determine that the peer does indeed host the file. If thisis true, then i will connect to the servant and download the file.

Web services do not currently provide reputation data and are notaccommodated in reputation management systems. There is no way formultiple copies of web services to determine which result, when resultsamong the web services differ, is correct. Moreover, there is nomechanism for determining which web services are important enough ordesirable enough to be run as identical copies across multiple computingdevices.

In view of the foregoing, there is a need for systems and methods thatovercome these deficiencies.

SUMMARY OF THE INVENTION

The following summary provides an overview of various aspects of theinvention. It is not intended to provide an exhaustive description ofall of the important aspects of the invention, nor to define the scopeof the invention. Rather, this summary is intended to serve as anintroduction to the detailed description and figures that follow.

The present invention is directed to web services, residing onassociated computing devices, each having an associated reputation. Aclient may desire to access the web service having the highest or bestreputation to be ensured of a greater degree of accuracy and confidence.The client calls the search service, passing in a contract thatrepresents the requested service behaviors, and attaches to whicheverhas the highest reputation.

According to aspects of the invention, multiple copies of web servicesmay vote amongst themselves on the results in the event that one or moreof the services starts giving incorrect or otherwise inconsistentresults. More particularly, the web services may run in lock-step witheach other, starting from the same original state. Such web servicesdesirably process identical protocols, and vote amongst themselves onthe results of each protocol response back to the calling service.Desirably, an odd-numbered copy of services is used in scenariosinvolving voting, in order to prevent voting deadlocks or ties.

According to further aspects of the invention, by combining this votingwith reputation data associated with each copy of the web service, aservice's reputation can be dynamically adjusted based upon howfaithfully it computes the results of work items (e.g., messages) sentto it. Once a web service on a particular node passes a certainreputation threshold (meaning that the web service's reputation hasdecreased beyond a predetermined acceptable level), the system can failover to healthy node, shun the poorly behaving service, cut off its lifesupport (e.g., scheduling, memory management), kill it, and/or restartthe service on a known good node.

Further aspects of the invention involve determining which servicesshould have multiple copies made and run on multiple computing devices.The worthy or important services may be dynamically determined atruntime based upon an observation of the resources the service requests,plus a notion of relative service priority.

Additional features and advantages of the invention will be madeapparent from the following detailed description of illustrativeembodiments that proceeds with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing summary, as well as the following detailed description ofpreferred embodiments, is better understood when read in conjunctionwith the appended drawings. For the purpose of illustrating theinvention, there is shown in the drawings exemplary constructions of theinvention; however, the invention is not limited to the specific methodsand instrumentalities disclosed. In the drawings:

FIG. 1 is a block diagram showing an exemplary computing environment inwhich aspects of the invention may be implemented.

FIG. 2 is a block diagram illustrating a computer system divided intothree component groups: the hardware component, the operating systemcomponent, and the applications programs component;

FIG. 3A is a block diagram illustrating a distributed operating systemfor creating unity among a multiplicity of devices, content,applications, and people, that may be used in accordance with thepresent invention;

FIG. 3B is a block diagram illustrating two operating system servicescommunicating with one another in accordance with the present invention;

FIGS. 3C and 3D are unilateral contracts associated with services inaccordance with the present invention;

FIG. 4 is a block diagram generally representing services communicatingwith one another over a web services application protocol in accordancewith the present invention;

FIG. 5 is a block diagram of an exemplary web service system inaccordance with the present invention;

FIG. 6 is a flow diagram of an exemplary method of determining which webservice to attach to in accordance with the present invention;

FIG. 7 is a flow diagram of an exemplary method of voting and reputationadjustment among web services in accordance with the present invention;and

FIG. 8 is a flow diagram of an exemplary method of determining whether aservice should be executed in multiple copies in accordance with thepresent invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

The subject matter is described with specificity to meet statutoryrequirements. However, the description itself is not intended to limitthe scope of this patent. Rather, the inventors have contemplated thatthe claimed subject matter might also be embodied in other ways, toinclude different steps or combinations of steps similar to the onesdescribed in this document, in conjunction with other present or futuretechnologies. Moreover, although the term “step” may be used herein toconnote different elements of methods employed, the term should not beinterpreted as implying any particular order among or between varioussteps herein disclosed unless and except when the order of individualsteps is explicitly described.

Overview

Multiple copies of web services reside on associated computing devices,each having an associated reputation. A client may desire to access theweb service having the highest or best reputation to be ensured of agreater degree of accuracy and confidence. The client calls the searchservice, passing in a contract that represents the requested servicebehaviors, and attaches to whichever web service has the highestreputation. Multiple copies of web services may vote amongst themselveson the results in the event that one or more of the services startsgiving incorrect or otherwise inconsistent results. Combining the votingwith reputation data associated with each copy of the web service allowsa service's reputation to be dynamically adjusted based upon howfaithfully it computes the results of work items sent to it.

Exemplary Computing Environment

Numerous embodiments of the present invention may execute on a computer.FIG. 1 and the following discussion are intended to provide a briefgeneral description of a suitable computing environment in which theinvention may be implemented.

Those skilled in the art will appreciate that the invention may bepracticed with other computer system configurations, including handhelddevices, multiprocessor systems, microprocessor based or programmableconsumer electronics, network PCs, minicomputers, mainframe computersand the like. The invention may also be practiced in distributedcomputing environments where tasks are performed by remote processingdevices that are linked through a communications network.

As shown in FIG. 1, an exemplary general purpose computing systemincludes a conventional personal computer 20 or the like, including aprocessing unit 21, a system memory 22, and a system bus 23 that couplesvarious system components including the system memory to the processingunit 21. The system bus 23 may be any of several types of bus structuresincluding a memory bus or memory controller, a peripheral bus, and alocal bus using any of a variety of bus architectures. The system memoryincludes read only memory (ROM) 24 and random access memory (RAM) 25. Abasic input/output system 26 (BIOS), containing the basic routines thathelp to transfer information between elements within the personalcomputer 20, such as during start up, is stored in ROM 24.

The personal computer 20 may further include a hard disk drive 27 forreading from and writing to a hard disk, not shown, a magnetic diskdrive 28 for reading from or writing to a removable magnetic disk 29,and an optical disk drive 30 for reading from or writing to a removableoptical disk 31 such as a CD-ROM or other optical media. The hard diskdrive 27, magnetic disk drive 28, and optical disk drive 30 areconnected to the system bus 23 by a hard disk drive interface 32, amagnetic disk drive interface 33, and an optical drive interface 34,respectively. The drives and their associated computer readable mediaprovide nonvolatile storage of computer readable instructions, datastructures, program modules and other data for the personal computer 20.

Although the exemplary environment described herein employs a hard disk,a removable magnetic disk 29 and a removable optical disk 31, it shouldbe appreciated by those skilled in the art that other types of computerreadable media which can store data that is accessible by a computer,such as magnetic cassettes, flash memory cards, digital video disks,Bernoulli cartridges, random access memories (RAMs), read only memories(ROMs) and the like may also be used in the exemplary operatingenvironment.

A number of program modules may be stored on the hard disk, magneticdisk 29, optical disk 31, ROM 24 or RAM 25, including an operatingsystem 35, one or more application programs 36, other program modules 37and program data 38. A user may enter commands and information into thepersonal computer 20 through input devices such as a keyboard 40 andpointing device 42. Other input devices (not shown) may include amicrophone, joystick, game pad, satellite disk, scanner or the like.These and other input devices are often connected to the processing unit21 through a serial port interface 46 that is coupled to the system bus,but may be connected by other interfaces, such as a parallel port, gameport or universal serial bus (USB). A monitor 47 or other type ofdisplay device is also connected to the system bus 23 via an interface,such as a video adapter 48. In addition to the monitor 47, personalcomputers typically include other peripheral output devices (not shown),such as speakers and printers. The exemplary system of FIG. 1 alsoincludes a host adapter 55, Small Computer System Interface (SCSI) bus56, and an external storage device 62 connected to the SCSI bus 56.

The personal computer 20 may operate in a networked environment usinglogical connections to one or more remote computers, such as a remotecomputer 49. The remote computer 49 may be another personal computer, aserver, a router, a network PC, a peer device or other common networknode, and typically includes many or all of the elements described aboverelative to the personal computer 20, although only a memory storagedevice 50 has been illustrated in FIG. 1. The logical connectionsdepicted in FIG. 1 include a local area network (LAN) 51 and a wide areanetwork (WAN) 52. Such networking environments are commonplace inoffices, enterprise wide computer networks, intranets and the internet.

When used in a LAN networking environment, the personal computer 20 isconnected to the LAN 51 through a network interface or adapter 53. Whenused in a WAN networking environment, the personal computer 20 typicallyincludes a modem 54 or other means for establishing communications overthe wide area network 52, such as the internet. The modem 54, which maybe internal or external, is connected to the system bus 23 via theserial port interface 46. In a networked environment, program modulesdepicted relative to the personal computer 20, or portions thereof, maybe stored in the remote memory storage device. It will be appreciatedthat the network connections shown are exemplary and other means ofestablishing a communications link between the computers may be used.

While it is envisioned that numerous embodiments of the presentinvention are particularly well-suited for computerized systems, nothingin this document is intended to limit the invention to such embodiments.On the contrary, as used herein the term “computer system” is intendedto encompass any and all devices comprising press buttons, or capable ofdetermining button presses, or the equivalents of button presses,regardless of whether such devices are electronic, mechanical, logical,or virtual in nature.

As illustrated in the block diagram of FIG. 2, a computer system 200 canbe roughly divided into three component groups: the hardware component202, the operating system component 204, and the applications programscomponent 206.

In certain computer systems 200, and referring back to FIG. 1, thehardware 202 may comprise the central processing unit (CPU) 21, thememory (both ROM 24 and RAM 25), the basic input/output system (BIOS)26, and various input/output (I/O) devices such as a keyboard 40, amouse 42, a monitor 47, and/or a printer (not shown), among otherthings. The hardware component 202 comprises the basic resources for thecomputer system 200.

The applications programs component 206 comprises various softwareprograms including but not limited to compilers, database systems, wordprocessors, business programs, videogames, and so forth. Applicationprograms provide the means by which computer resources are utilized tosolve problems, provide solutions, and process data for various users(e.g., machines, other computer systems, and/or end-users).

The operating system component 204 comprises the operating system itselfand its shell and kernel. An operating system (OS) is a special programthat acts as an intermediary between application programs and computerhardware, and the purpose of an operating system is to provide anenvironment in which a user can execute application programs. The goalof any operating system is to make the computer system convenient touse, as well as utilize the computer hardware in an efficient manner.

The operating system is generally loaded into a computer system atstartup and thereafter manages all of the application programs (orsimply “applications”) in the computer system. The application programsinteract with the operating system by requesting services via anapplication program interface (API). Some application programs enableend-users to interact with the operating system via a user interfacesuch as a command language or a graphical user interface (GUI).

An operating system traditionally performs a variety of services forapplications. In a multitasking operating system where multiple programsmay be running at the same time, the operating system determines whichapplications should run in what order and how much time should beallowed for each application before switching to another application fora turn. The operating system also manages the sharing of internal memoryamong multiple applications, and handles input and output to and fromattached hardware devices. The operating system also sends messages toeach application (and, in certain cases, to the end-user) regarding thestatus of operations and any errors that may have occurred.

An operating system's shell is the interactive end-user interface to anoperating system. A shell is the outer layer of an operating system thatis directly accessible by application programs and even directly byend-users. In contrast to a shell, the kernel is an operating system'sinnermost layer that interacts directly with the hardware components.

Distributed Operating System

A distributed operating system 302 is illustrated in FIG. 3A. Thedistributed operating system participates in a noncentralized networkconsisting of numerous computer systems that can communicate with oneanother and that appear to users as parts of a single, large, accessible“storehouse” of shared hardware, software, and data. The distributedoperating system 302 is conceptually the opposite of a centralized, ormonolithic, operating system in which clients connect to a singlecentral computer, such as a mainframe. The power of control andcoordination of the distributed operating system 302 comes not frombeing at one place at one time but instead comes from being capable ofan omnipresence to compose services, local or remote, and formapplications that are desired by users.

The distributed operating system 302 creates unity from multiplicity.The multiplicity includes devices 304, which include any piece ofequipment or mechanism designed to serve a special purpose or perform aspecial function, such as a personal digital assistant, a cellularphone, or a monitor display, among others. The multiplicity alsoincludes any piece of content 306, such as sound, graphics, animation,video, or other pieces of data or information. The multiplicity furtherincludes applications 308, which are programs designed to assist in theperformance of a specific task, such as word processing, accounting, orinventory management. Applications 308 are compositions of one or moreservices. The multiplicity yet further includes people 310. The people310 are those individuals wishing to gain access to the distributedoperating system 302 to use resources, such as devices 304, pieces ofcontent 306, and applications 308.

Devices 304, content 306, applications 308, and people 310 can beabstracted as autonomous computation entities called services thatexchange messages according to protocols, which are defined by eachservice. Services can be local to a computer system but can also belocated at a remote computer system. Services can be accessed through asingle trust domain but can also be accessed through another trustdomain with its own security policy. Services can be discoverablethrough a directory service but can also be discovered by services thatare not directory services.

The distributed operating system 302 unifies devices 304, content 306,applications 308, and people 310 to create a computing environment forcomposing services, and allows the discovery of services and thecomposition of services. Devices 304, content 306, applications 308, andpeople 310 are loosely coupled to the distributed operating system 302.Yet, the distributed operating system 302 can compose, arrange, orcombine various pieces of the multiplicity. Each piece of themultiplicity 304-310 need not be known a priori by the distributedoperating system 302, but each piece is preferably discoverable so thatthe distributed operating system 302 can compose, arrange, or combine tocreate the desired functionality. This unifying effect of thedistributed operating system 302 allows every piece in the multiplicityto know how to communicate to every other one regardless of how diverseone piece of the multiplicity might be. Because devices 304, content306, applications 308, and people 310 can be unified, each of them canbe located locally or dispersed remotely and yet all of them cancommunicate with one another.

FIG. 3B illustrates two operating system services 310A, 310B, each witha port identifiable by a uniform resource indicator (URI) 310A-1,310B-1, which constitutes a unique designation of an operating systemservice or another service, and a unilateral contract 310A-2, 310B-2.Several primitives are described: a designation primitive, whichcomprises a port, such as the ports identifiable by the URI, 310A-1,310B-1; a behavioral primitive, which comprises the unilateral contract,such as unilateral contracts 310A-2, 310B-2; an organizationalprimitive, which comprises a service, such as operating system services310A, 310B; and a communication primitive, which includes a set of verbs362 known by all services, that separates the data plane from thecontrol plane for facilitating communication of control information anddata information. The term “verb” means the inclusion of commanding,instructing, ordering, calling, controlling, requesting, or managing aservice to perform a certain task. The set of verbs 362 is essentially aprotocol for expressing behaviors for services running on a distributedoperating system.

These primitives are capable of being applied at various levels, such asa retrogression to a less complex level of organization or a progressionto a more complex level of organization: at a file containing a piece ofcontent 306; at a device among devices 304, which can be either internalor external to a computer system; at an application among applications308; at a computer system; across a home or an office; across an entireneighborhood or multiple offices of an organization; and across theentire world. This retrogression and progression is made possible by theuse of a combination of these primitives everywhere.

Ports of services are endued with behavioral types, which are specifiedby the unilateral contracts. An exemplary communication mechanism of thedistributed operating system 302 is through programmatically wiredports. Wired ports are possible if the behavior type of one port (of aservice) is compatible with the behavior type of another port (ofanother service). When ports are programmatically wired to each other,which are identifiable by URIs 310A-1, 310B1, operating system services310A, 310B communicate by sending messages to each other. Unilateralcontracts 310A-2, 310B-2 are expressed in a language specifying an orderof messages which flow in or out of operating system services 310A,310B. By the use of messages, resources distributed in multiple trustdomains, each with its own security policy, can communicate with oneanother.

Sharing of resources is possible through interaction in a compatible waywith the behaviors of the resources. Behaviors of resources (representedby services) are expressed in unilateral contracts. For example, a filecan be a service. A service can be regulated by a unilateral contract.Thus, one can attach behavioral conditions to files via unilateralcontracts to govern access control. A read-only file should behavedifferently from a file available for both reading and writing. It ispreferred to represent each file type through separate unilateralcontracts. A read-only file unilateral contract may include thefollowing behavioral expression: REC F (read.F+drop).0, whereas aread-write file's unilateral contract has the following behavioralexpression: REC F (read.F+write.F+drop).0. In parsing the behavioralexpressions, the term REC F indicates a recursion on a behavior phraseF; the behavior phrase F indicates the behavior expressions inside thepairs of parentheses; the verb “read” indicates a read operation; theperiod symbol “.” denotes a sequence in which the behavior phrase beforethe period symbol occurs and after which the behavior phrase followingthe period symbol will then occur; the plus sign symbol “+” indicates achoice between one or more behavior phrases; the verb “write” indicatesa write operation; the verb “drop” indicates the termination of thecommunication between two services; and the zero symbol “0” denotes thetermination of the behavior expression.

A portion of the unilateral contract 310A-2 is illustrated in FIG. 3C.Line 310A-3 contains the key word UNILATERALCONTRACT followed by thedesignator “OSSERVICE,” and a pair of open and closed curly brackets “{}” for delimiting the definition of the unilateral contract 310A-2. Line310A-4 declares the signature of the OPEN operation that takes a filename “FILENAME” as a parameter. To use the operating system service310A, external services specify a name of a file to be opened via theOPEN operation. Thus, the OPEN operation should be the first operationthat is invoked by other services for each session. The PLAY operationis declared on line 310A-5. The PLAY operation takes another service'sport as a parameter. When the PLAY operation is invoked by otherservices, the operating system service 310A reads a stream of data froman open file and transmits the read data toward the given service'sport. Other services, such as the operating system service 310B, canalso record information to opened files via the RECORD operation, whichis declared on line 310A-6. The RECORD operation takes data as aparameter. This data is written by the RECORD operation to the openedfile. When all desired operations have been carried out on the openedfile, the opened file can be closed via the CLOSE operation, which isdeclared on line 310A-7. The CLOSE operation takes a file name“FILENAME” as an argument so that the CLOSE operation knows which fileto close.

Lines 310A-8-310A-9 contain the behaviors of the operating systemservice 310A. Line 310A-8 contains a behavior sentence: B=OPEN.BPR,where B is a behavior rule; OPEN denotes that the OPEN operation is thefirst operation to be invoked in using the operating system service310A; the period “.” denotes that additional behaviors are to follow theinvocation of the OPEN operation; BPR refers to a second behaviorsentence defined further on line 310A-9. Line 310A-9 contains thefollowing behavioral sentence: BPR=PLAY.BPR+RECORD.BPR+CLOSE, where BPRdenotes the second behavior; PLAY.BPR denotes the invocation of the PLAYoperation, which is then followed by the second behavior again (arecursion); RECORD.BPR denotes the invocation of the RECORD operation,which is then followed, recursively, by the second behavior; CLOSEdenotes the invocation of the CLOSE operation; and the plus signs “+”denote choices that other services, such as the operating system service310B, can make to invoke among the PLAY operation, the RECORD operation,or the CLOSE operation.

A portion of the unilateral contract 310B-2 is illustrated in FIG. 3D.Line 310B-3 contains the keyword UNILATERALCONTRACT followed by thedesignator “OSSERVICE,” and a pair of open and closed curly brackets “{}” for delimiting the definition of the portion of the unilateralcontract 310B-2. Line 310B-4 declares the signature of the OPENoperation that takes a file name “FILENAME” as a parameter. The PLAYoperation is declared on line 310B-5. The PLAY operation takes anotherservice's port as a parameter. The CLOSE operation is declared on line310B-6 and it takes a filename “FILENAME” as an argument so that theCLOSE operation knows which file to close.

Lines 310B-7-310B-8 contain the behaviors of the operating systemservice 310B. Line 310B-7 contains a behavior sentence: B=OPEN.BP, whereB is a behavior rule; OPEN denotes that the OPEN operation is the firstoperation to be invoked in a session with the operating system service310B; the period “.” denotes that the additional behaviors are to followthe invocation of the OPEN operation; and BP refers to a second behaviorsentence defined further on line 310B-8. Line 310B-8 contains thefollowing behavioral sentence: BP=PLAY.BP+CLOSE, where BP denotes thesecond behavior; PLAY.BP denotes the invocation of the PLAY operation,which is then followed by the second behavior again (a recursion); CLOSEdenotes the invocation of the CLOSE operation; and the plus sign “+”denotes choices that an external service, such as the operating systemservice 310A, can make to invoke among the PLAY operation and the CLOSEoperation.

The unilateral contract 310A-2, when accepted by the operating systemservice 310B, and the unilateral contract 310B-2, when accepted by theoperating system service 310A, creates an instance of communicationbetween the operating system service 310A and the operating systemservice 310B. Each unilateral contract 310A-2, 310B-2 cannot be acceptedby the operating system services 310A, 310B by a mere promise toperform, but preferably by the performance of unilateral contracts310A-2, 310B-2 in accordance with the behaviors expressed in thoseunilateral contracts. Thus, if the operating system service 310Bcomplies with and performs the behaviors as expressed by behaviorsentences 310A-8, 310A-9 of the unilateral contract 310A-2, theoperating system service 310B is bound to provide the promised services.For example, if the operating system service 310B has performed by firstinvoking the OPEN operation as specified by the behavioral sentence310A-8 and then either invoking the PLAY operation or the RECORDoperation or the CLOSE operation as specified by the behavioral sentenceshown on line 310A-9, then the operating system service 310A complieswith the requested invocations to provide the desired services, such asopening a file, playing the content of the file, recording content intoa file, or closing the file.

Web Service Application Protocol and SOAP Processing Model

An exemplary protocol for using web services is the web servicesapplication protocol (WSAP), that, among other things, formallyrepresents the state and behavior of a service via a port (e.g., URI),whereby the entities in a service may be contacted directly forinteraction. In other words, unlike other protocols, WSAP describes thebehaviors of a web service in a formalized manner, e.g., with eachbehavior of a service having its own port. The WSAP protocol alsofacilitates the usage of intermediaries by defining shared semantics onverbs, which identify what a message is, what the processing of messageis, and how it affects the state/behavior of the receiver. Further, theordering of messages and operations provides context to what isoccurring.

In general, anytime the behaviors (or states) of a service's logicalentities diverge, WSAP allows the service to give each behavior/stateits own port (e.g., an identity such as URI). In this way, the clientcan communicate directly with the logical entity within the service thatcontrols the behavior/state. By way of example, consider a web servicethat provides access to a storage device such as a hard disk drive.Traditionally, after opening a file via a file system and receiving afile handle, to read and write data to the open file, the client wouldsend the file handle to the file system. With WSAP, instead of returninga file handle, an arbiter service returns a URI to another logicalentity (another service) of the web service that provides access to thebehavior (the operation to perform) on the appropriate state (the data).In this example, with WSAP the client would receive a port for writing(e.g., using a suitable XML schema) to allowed locations on the storagemedium itself, and another port for reading from the storage medium,without again going through the arbiter. Note that a read operationcorresponds to one behavior of a storage medium, and thus may have adifferent URI from a write operation, which corresponds to anotherbehavior. As can be appreciated, this model of data access is moreefficient than going to a file system controller for each operation,particularly if the file system would otherwise be a bottleneck.

Moreover, in addition to the WSAP's clear interaction points in the formof ports that expose state and behavior, WSAP messages use sharedsemantics that describe what can be done on the interaction point. Thesemantics comprise shared message verbs that are separate from thepossibly proprietary) message details, allowing any arbitraryintermediary to observe, to a known extent, what is taking place, whileleaving the message contents secure.

Message ordering can also serve to provide context to an observer. Forexample, each service may define which WSAP operations it accepts, andin which order. Further, the communication with each port ordinarilyidentifies the behavior that is being requested, since each behavior maybe assigned its own port. These interaction patterns between serviceports, which in fact may be maintained as a list of ports, provide areadily observable directory of what WSAP operations occurred, and inwhat order they occurred.

As a result, an intermediary can process exchanged messages to add valueto communications. For example, when it is known from the requestedoperation type and/or known URI that a message will not change the stateof a resource, an intermediary such as a cache service can safely readdata from its own cache to satisfy the request, rather than invalidatethe cache and pass the message (to another service) to access otherstorage media. Note that the intermediate cache service need only knowthe operation type from the protocol's semantics, or the URI if known tobe to a read-only service, and not the internal contents of the message,which may remain secure (and indeed may have been encrypted via anotherintermediary service). Examples of such read-like operations in WSAP inwhich no state changes are involved include operations referred to usingthe shared semantics of QUERY or GET. Such operations each have thecharacteristic of being safe, essentially meaning read-only access issufficient to satisfy the operation request, and idempotent, essentiallymeaning that the same operation can be repeated indefinitely withoutchanging state. Note that a cache is only one example of such anintermediary, and an intermediary can be any other service asappropriate, e.g., a load balancer, data compressor, replicationservice, encryption/decryption mechanism, and so forth.

Moreover, because of behaviors having their own ports, a SOAP processingmodel is provided that defines how to compose multiple web services torun in parallel or sequentially. The model allows the implementation ofseveral services, such as the SOAP-aware serializer, to serialize ordeserialize the header and body of a given SOAP message concurrently.This allows services that depend upon parts of the SOAP envelope tounblock whenever a header that a service cares about is available. Notethat when two communicating parties are co-located, e.g., on the samedevice, the forwarding and transport services do not serialize the SOAPenvelope for performance reasons.

In a distributed programming model, a basic programming primitive is alightweight port, comprising an unordered collection of messages thatcan be used to store state and effect communication. Items are stored byreference, and ports support (at least) two primitive operations: Post() and Get( ). Posting an item, such as a class instance, buffers thereference and returns immediately. Retrieving an item based on asupplied key can block if no item with the key has been posted by theremote service. Although ports can be used directly by services, theyare typically used within the context of a port collection thatassociates a specific message type with each port in the collection.Port collections can be created with a fixed or a dynamic set ofsupported message types depending on the flexibility needed, but from aprogrammer's point of view, a port collection is simply a typed port.

Using strictly port communication, service code can implement criticalsections and complicated I/O protocols and produce a very concurrentprocessing model. To this end, as described further below, WSAPcommunicates with a web service, in which a client consumer requests aserver provider to perform some service for the client, and provide theclient with an appropriate response. As will be understood, however, thegeneral web service model is not limited to a server running softwarefor a client, but applies to any resource that a client wants to access,e.g., the uniform service model supported by WSAP can be used to modelfiles, devices, services, and people. For example, hardware may becomponentized to an extent, and in many ways will be virtuallyindistinguishable from software-oriented web services, in that a usermay select a set of hardware components and interconnect them to performa computing task. As long as each device obeys the communicationprotocols (and the appropriate amount of bandwidth is available),virtually any authorized device will be able to communicate with anotherdevice to use its resources.

FIG. 4 represents an architecture 400 comprising web client code 402(e.g., which may be a computer system, such as that described withrespect to FIG. 1) communicating with a web service 404 via WSAPmessages communicated within a SOAP (simple object access protocol)envelope. The identity of a service is described by a named port (e.g.,a URI), and the state of a service is typically described by an XMLschema. Note that the service may be local (within the same machine asthe client code) or may be a remote resource such as a server or ahardware device. In general, WSAP defines, semantically, what a webservice does, while the SOAP processing model defines how to composemultiple web services to run in parallel or sequentially, and may beused in accordance with the present invention to compose internalservices to a computing device in a manner that is substantiallyidentical to external web services, e.g., via in-memory non-XMLserialized data.

As with other web services, WSAP-capable services operate by offering acontract in response to a contract request, as represented in FIG. 4 bythe web service contract request 406 and contract offering 408. Notethat in WSAP, to obtain a contract, a port is exposed to the clientcorresponding to a LOOKUP operation. Binding list information isprovided with the response 408, e.g., as an identifier/port thatcorresponds to an array of key/value pairs that “instance” the contract.Services are not required to implement WSAP operations other than theLOOKUP operation, although when the semantics of an interaction iscompatible with a WSAP operation, the WSAP service should implement theWSAP operation. In providing a service, the service author may map anyinteraction or service functionality using WSAP verbs andrepresentations. Not all WSAP verbs have to be used for a particularservice, but instead a subset may be used, depending on the scenario.

WSAP is a SOAP-based protocol useful in a distributed computingenvironment that defines a web service in terms of identity, state, anda set of operations that can be used according to the behavior of theservice. This effectively enables services and their state to beoperated upon in a consistent and observable manner. In WSAP, abehavioral web service is an abstract notion of a process that has anidentity and a behavior. This concept is abstract in the sense that itcan apply to many different application models and namespaces. WSAPdefines one such application model which has been designed to scale tothe internet at large, but many other application models are possible.

The notion of identity in WSAP is similar to that of a resourceidentified by a URI in the traditional web model, namely anything thathas identity. The purpose of identity is to enable a service todifferentiate itself from other services that may exist within the samenamespace. The scope of WSAP web services may be the internet, whichmeans that DNS or IP is the naming authority for the URI “authority”component, although in more a limited scope such as within a device orlimited network, another naming authority is feasible.

Each WSAP web service has a unique identity or URI. As with any otherURI (such as on the web), no semantic knowledge is obtained based on theURI alone, and similarly, a relationship (e.g., a containmentrelationship) between two WSAP web services cannot be deduced from theirURIs. In WSAP, URIs may be either relative or absolute. If a URI isrelative then its base is determined by XML Base. WSAP does not placeany limit on the length of a URI, and thus any WSAP receiver needs to beable to handle the length of any URI that it publishes; WSAPimplementations should be able to deal with URIs of at least twokilobytes in length.

The interaction between services (which can be thought of as a protocol)is the behavior of a service, and can be formally described as abehavioral type, with the contract being the description of a serviceusing behavioral types. A service can be implemented in any number ofways, as long as it conforms to the contract it claims to implement.Note that in WSAP, a contract is a service like essentially everythingelse, whereby contracts are identified by ports (e.g., URIs) and can beoperated on as any other service, according to its contract.

The notion of behavior is based on how a service interacts with itsenvironment in terms of the sequence of message types that it generatesand accepts. Behavior enables a service to differentiate its particularbehavior from the behavior of other services that may exist within thesame namespace, and behavior is relative to the set of possiblebehaviors defined for a particular namespace in which the service isdefined. In other words, behavior is a distributed state machine exposedby the service. Thus, by including the interactions of those resourcesor services with other services, behavior augments the current web modelfor identifying resources. Two services need to have compatiblebehaviors to communicate; that is, they must compose. If the behavior ofa service is not known by another service, this implies that the twoservices cannot interact, which is similar to the web model of aresource wherein a URI cannot be dereferenced because the mechanism fordoing so is unknown to the holder of that URI.

Exemplary Embodiments

The exemplary embodiments described herein can be used in a distributedoperating system, such as the one described above, and use a protocol,such as WSAP, described above, though the invention can be implementedin any appropriate system using any appropriate protocol, and is notlimited to a distributed operating system and/or WSAP.

A web service in accordance with the present invention may comprisecomponents such as a location, a behavior, and semantics. For example, aservice may include a designation primitive, a behavioral primitive thatcomprises a unilateral contract, and a communication primitive. Anexemplary system may include a distributed operating system fororchestrating the services executing on a computer system so as tocontrol and coordinate resources.

Exemplary web services include a data service and a stateless controlservice. For example, a data service and a stateless control service mayrun on a single node, such as a computing device. A block diagram of anexemplary system with nodes and web services is shown in FIG. 5. Asshown, a client 500 sends a work item or request to the computingdevices (nodes) 510, 520, 530 via a calling service. The client 500 maysend the work item or request directly to each of the computing devices510, 520, 530, or via an intermediary (not shown).

Each computing device preferably comprises a stateless control serviceand a data service, though other services and arrangements arecontemplated, and the general term “web service” is shown in FIG. 5.There is a relationship between the stateless control service and thedata service on each computing device. A client calling service may callthe stateless control service, on a computing device, which then mayaccess the data service to provide requested information, for example,to the client calling service. A second computing device, remote fromthe first computing device, may also comprise a data service and astateless control service. Another computing device may desire to havean identical copy of the data service to that of the first computingdevice, in which case the second data service may subscribe to the firstdata service's updates. In an exemplary case of multiple computingdevices, each comprising a web service, it is contemplated that eachwill have subscriptions to the others, so that they are in sync by,e.g., publish/subscribe mechanisms. Thus, the web services 515, 525, 535residing on the various computing devices 510, 520, 530 are desirablymultiple, identical copies.

Each web service, residing on an associated computing device, may havean associated reputation. Although the web services should be identical,and hence have identical reputations, there may be instances in whichthis is not the case. For example, a web service may be affected by avirus, in which case, it gives incorrect results, or results that arenot in sync with what the other web services would generate. Thus, aclient may desire to access the web service having the highest or bestreputation to be ensured of a greater degree of accuracy and confidence.Thus, the client 500 may check the reputation of each potential webservice, and then select which one to attach to, or select which one'sresult to use (e.g., the client 500 will use the result of the webservice having the highest reputation).

FIG. 6 is a flow diagram of an exemplary method of determining which webservice to attach to in accordance with the present invention. Theclient starts a dialogue with each of the multiple, identical webservices, at step 600, that has a capability of fulfilling the client'swork item or request. At step 610, the client analyzes the reputation ofeach web service to determine and select which has the highestreputation (step 620). This is desirable, for example, when thecomputing devices are not communicating with each other, in which casetheir services may not be in sync with each other. The client thenattaches to that web service having the highest reputation, at step 630,or otherwise ultimately uses the result provided by that web service. Itis noted that after the caller sends the request to analyze thereputation of each web service, the caller is desirably able to performother work while the request is being satisfied.

As described herein, multiple, identical copies of desired web services(e.g., web services 515, 525, 535) may be run across computing devices(e.g., devices 510, 520, 530 in a distributed operating system) toprovide reliability in the face of failure in any one node (which may beconsidered to be resident on, or coincident with, an associatedcomputing device). By running multiple copies of these services, theservices may vote amongst themselves on the results in the event thatone or more of the services starts giving incorrect or otherwiseinconsistent results. Such results can be from sickness in theunderlying hardware, or via a malicious virus infecting one of thenodes, for example. Desirably, an odd-numbered copy of services is usedin scenarios involving voting, in order to prevent voting deadlocks orties. However, other methods of determining the preferred result may beused, such as considering additional factors like the reputation of theparticular web service, how long the particular web service has beenactive, and how many inward links exist to the particular web service.

FIG. 7 is a flow diagram of an exemplary method of voting and reputationadjustment among web services in accordance with the present invention.At step 700, each web service determines its result (e.g., the result toa client's work item or request). At step 710, the results are tabulatedor otherwise analyzed to determine which result is prevalent. Becausethe copies of the web services are supposedly identical, each resultshould be identical. However, for various reasons, such as a virus ormalfunctioning software or hardware, the results may not be identicalacross all the copies of the web service being run on the variouscomputing devices.

The result that is most prevalent is determined to be the accurate or“correct” result, at step 720, and this result is reported back to thecalling party or client device. For example, if web service 515 and 525each determined the result to a client's request to be “1” and if webservice 535 determined the result to the same request to be “2”, theresult of “1” would be most prevalent (two result occurrences of “1”versus one result occurrence of “2”), and this result would be reportedback to the client.

By combining this voting with reputation data associated with each copyof the web service, a service's reputation can be dynamically adjusted,at step 730, based upon how faithfully it computes the results of workitems (e.g., messages) sent to it. For example, a service which has beentaken over by a virus may still have the desired cryptographic keys tocommunicate to its brethren nodes; however, any variance from itscontract will desirably result in a decrement to its reputation.Continuing with the example in which web service 515 and 525 eachdetermined the result to a client's request to be “1” and web service535 determined the result to the same request to be “2”, because webservice 535 got the less popular, and hence incorrect, answer, thereputation of web service 535 is decreased. Moreover, the reputations ofweb services 515 and 525 may be increased because they got the correctresult. It is contemplated that other factors may affect a web service'sreputation, such as responsiveness, latency, and uptime.

Once a web service on a particular node passes a certain reputationthreshold (meaning that the web service's reputation has decreasedbeyond a predetermined acceptable level), the system can fail over tohealthy node, shun the poorly behaving service, cut off its life support(e.g., scheduling, memory management), kill it, and/or restart theservice on a known good node. Thus, a trustworthy system can be createdfrom multiple untrusted sources. With respect to FIG. 7, after the webservices have had their reputations adjusted accordingly, it isdetermined at step 740 if any of the web services have reputations thatnow exceed a threshold. If so, then remedial action is taken, at step750 (e.g., fail over, restart the web service on another computingdevice, etc.).

Thus, each of the web services, when in communication with each other,may check and compare their respective results with each others'results. Based on these results, the services vote to determine whichresult is “correct”. Each service's reputation is then adjusted based onhow its result compared to the “correct” result. It is noted that apositive feedback loop is desirably created. A larger community ofservice reputations leads to greater overall trust, which attracts alarger community of services, which increases the overall trust in thecomposite system.

In a typical computing system, resources are scarce and must beallocated appropriately. Not every web service would be considered“worthy” or “important” enough to warrant providing and executingmultiple redundant copies across computing devices, in such a scarceresource environment. FIG. 8 is a flow diagram of an exemplary method ofdetermining whether a service should be executed in multiple copies inaccordance with the present invention. The services that are desirablyimplemented as multiple, identical copies may be dynamically determinedat runtime based upon an observation of the resources the servicerequests, plus a notion of relative service priority. Services whichtend to consume a lot of resources (e.g., relative port i/o activity,memory usage, network traffic, etc.) are desirably candidates for“importance”. Thus, if a service is not doing much, it may not be asimportant and worthy of replication as one which is consuming more CPUtime. Services which are truly “important” will tend to bubble-up inprioritization at runtime, and those become candidates for automaticreplication. While static system policy can be used to dictate how manyservices should be replicated, this scheme allows for automaticdiscovery of the correct services to replicate. This will also result ina biological-inspired set of fluid emergent behaviors which are noteasily predicted when the individual services are initially authored.

At step 800, a web service is monitored at runtime. It is determined howmany resources the service requests at step 810, and how many inwardlinks exist to the web service, at step 820. The results of steps 810and 820 are considered and compared to a threshold value, at step 830.Based on the comparison to the threshold value, it is determined whetheror not to create multiple redundant copies of the web service. If so,then the multiple redundant copies are created on various computingdevices, at step 840.

Services can be intermediaries for other services. A first tier ofservices may be controlled by a second tier of services, for example.Thus, multiple tiers of services may be implemented into an exemplarycomputing system. The services on a given node compete for resources(memory and bandwidth) which are allocated by a local scheduler. Forexample, multiple first-tier nodes vote for an elected second-tierleader node which is responsible for allocating competing resourcesacross the multiple first-tier nodes. Depending upon the depth of theservice composition, the second-tier leaders may elect third tierleaders as desired.

CONCLUSION

The subject matter is described with specificity to meet statutoryrequirements. However, the description itself is not intended to limitthe scope of this patent. Rather, the inventors have contemplated thatthe claimed subject matter might also be embodied in other ways, toinclude different steps or combinations of steps similar to the onesdescribed in this document, in conjunction with other present or futuretechnologies. Moreover, although the term “step” may be used herein toconnote different elements of methods employed, the term should not beinterpreted as implying any particular order among or between varioussteps herein disclosed unless and except when the order of individualsteps is explicitly described.

The various systems, methods, and techniques described herein may beimplemented with hardware or software or, where appropriate, with acombination of both. Thus, the methods and apparatus of the presentinvention, or certain aspects or portions thereof may take the form ofprogram code (i.e., instructions) embodied in tangible media, such asfloppy diskettes, CD-ROMs, hard drives, or any other machine-readablestorage medium, wherein, when the program code is loaded into andexecuted by a machine, such as a computer, the machine becomes anapparatus for practicing the invention. In the case of program codeexecution on programmable computers, the computer will generally includea processor, a storage medium readable by the processor (includingvolatile and non-volatile memory and/or storage elements), at least oneinput device, and at least one output device. One or more programs arepreferably implemented in a high level procedural or object orientedprogramming language to communicate with a computer system. However, theprogram(s) can be implemented in assembly or machine language, ifdesired. In any case, the language may be a compiled or interpretedlanguage, and combined with hardware implementations.

The methods and apparatus of the present invention may also be embodiedin the form of program code that is transmitted over some transmissionmedium, such as over electrical wiring or cabling, through fiber optics,or via any other form of transmission, wherein, when the program code isreceived and loaded into and executed by a machine, such as an EPROM, agate array, a programmable logic device (PLD), a client computer, avideo recorder or the like, the machine becomes an apparatus forpracticing the invention. When implemented on a general-purposeprocessor, the program code combines with the processor to provide aunique apparatus that operates to perform the functionality of thepresent invention.

While the present invention has been described in connection with thepreferred embodiments of the various figures, it is to be understoodthat other similar embodiments may be used or modifications andadditions may be made to the described embodiments for performing thesame functions of the present invention without deviating therefrom.Therefore, the present invention should not be limited to any singleembodiment, but rather construed in breadth and scope in accordance withthe appended claims.

1. A web service system, comprising: a first computing device having afirst web service with a first reputation; and a second computing devicehaving a second web service with a second reputation; the first andsecond computing devices each adapted to: receive a request anddetermine a first and second result, respectively; determine a preferredresult based on the determined first and second results; adjust thefirst reputation if the first result is inconsistent with the preferredresult; and adjust the second reputation if the second result isinconsistent with the preferred result.
 2. The system of claim 1,wherein the second web service is a second, independent instance of thefirst web service.
 3. The system of claim 1, wherein the first webservice is de-activated if the first reputation exceeds a firstthreshold, and the second web service is de-activated if the secondreputation exceeds a second threshold.
 4. The system of claim 1, furthercomprising a client that sends the request to the first and secondcomputing devices, the client connected within a distributed operatingsystem.
 5. The system of claim 4, wherein the client polls the first andsecond web services for their respective reputations, and attaches toone of the web services based on their respective reputations.
 6. Amethod comprising: sending a request to a web service system, whereinthe web service system comprises a plurality of web services; receivingdata from each of the plurality of web services, wherein the dataindicates a respective reputation of each of the plurality of webservices; analyzing the respective reputations to determine a particularone of the plurality of web services having a sufficient reputation; andestablishing a connection with the particular one of the plurality ofweb services; wherein the method is implemented by a client device. 7.The method as recited in claim 6, wherein for each of the plurality ofweb services, the respective reputation is determined based, at least inpart, on a result generated by the web service in response to therequest.
 8. The method as recited in claim 6, wherein the web servicedetermined to have a sufficient reputation is the web service determinedto have the best reputation of the plurality of web services.
 9. Themethod as recited in claim 6, wherein a particular one of the pluralityof web services determined to have a reputation below a minimumthreshold is deactivated.
 10. The method as recited in claim 6, whereineach of the plurality of web services comprise a distinct copy of thesame web service.
 11. The method as recited in claim 6, wherein: a firstweb service of the plurality of web services generates a result inresponse to the request; a second web service of the plurality of webservices generates a result in response to the request; and therespective reputations of the first and second web services aredetermined based, at least in part, on a comparison of the resultsgenerated by the first and second web services in response to therequest.
 12. The method as recited in claim 6, wherein: each of theplurality of web services generates a result in response to the request;a correct result is determined; and the respective reputations of eachof the plurality of web services is determined based, at least in part,on a comparison of the respective results generated by each of theplurality of web services and the correct result.
 13. The method asrecited in claim 12, wherein, the correct result is determined to be theresult generated in response to the request by the greatest number ofthe plurality of web services.
 14. The method as recited in claim 12,wherein, the reputation is negatively adjusted for each of the pluralityof web services that generates a result in response to the request thatis not the correct result.
 15. The method as recited in claim 12,wherein, the reputation is positively adjusted for each of the pluralityof web services that generates a result in response to the request thatis equal to the correct result.
 16. One or more non-volatilecomputer-readable media comprising computer-executable instructionsthat, when executed, direct a web service system to: associaterespective reputations with each of a plurality of web services; receivea client request such that the client request is received by each of theplurality of web services; determine a correct response to the clientrequest; validate the respective reputations by comparing responses tothe client request from each of the plurality of web services with thecorrect response, such that for a web service providing a response tothe client request that does not match the correct response, therespective reputation is reduced; and establish a connection between theclient and a particular one of the plurality of web services, whereinthe particular one of the plurality of web services has a sufficientlypositive reputation.
 17. The one or more non-volatile computer-readablemedia as recited in claim 16, wherein each of the plurality of webservices comprises a distinct, independent copy of a single web service.18. The one or more non-volatile computer-readable media as recited inclaim 16, further comprising computer-executable instructions that, whenexecuted, direct the web service system to deactivate any of theplurality of web services for which the respective reputation is reducedto be below a reputation threshold.